Providing directional force feedback in free space

ABSTRACT

Technology is presented for generating directional force feedback in free space to a user using a mass rotatable about a movable axis. The mass and movable axis are responsive to a control signal to generate a force vector having a direction and a magnitude in three-dimensional space to provide feedback to the user. The force vector is created in response to an event in application executing in a processing device. The force vector represents feedback regarding an event in the application.

CROSS REFERENCE TO RELATED APPLICATION

This patent application has overlapping subject matter description withU.S. patent application Ser. No. 12/821,102, filed Jun. 22, 2010entitled “FREE SPACE DIRECTIONAL FORCE FEEDBACK APPARATUS” havinginventors Erik Tidemand, Clayton Chang, Muneeb Iqbal Karim, KentHuntsman, Alex Garden, filed concurrently herewith, and herebyspecifically incorporated by reference herein.

BACKGROUND

Applications controlled by users on processing devices provide sensoryfeedback to users in audio, visual and sensory forms. The applicationscan use the feedback to provide instructions and information, or as anavigational aid. Applications in the entertainment field, such asgames, strive to improve a user's experience of actually being in acomputer-generated reality. For example, hand held devices usingaccelerometers and other sensors allow user's physical motions to betranslated into a gaming application, and provide vibration feedbackbased on in-game events.

In virtual reality environments such as those used in games, a user caninteract with the virtual environment through an on-screenrepresentation of the user such as an avatar. Events which occur in thegame may be translated into feedback into a control device. For example,when a user hits a tennis ball using a movement based controller, thecontroller may vibrate.

SUMMARY

Technology is presented for generating directional force feedback infree space to a user. The technology uses a mass rotatable about amovable axis. The movable axis can be rotatable about an axis orthogonalto the movable axis, or be coupled to an axial structure and transversearm which positions the movable axis within a range of movement about apivot. The mass and movable axis are responsive to a control signal togenerate a force vector having a direction and a magnitude in threedimensional space to provide feedback to the user. The force vector iscreated in response to an event in application executing in a processingdevice. The force vector represents feedback regarding an event in theapplication.

The event makes generation of the meaningful in the context of theapplication. For example, the force to be generated can represent amessage to the user. In some examples, the message's meaning can be aninstruction or a physical response to a question or action of the user.In another example, the force to be generated can correspond to avirtual force vector being directed on a virtual object in the contextof the application, and the virtual object corresponds to the physicalobject which the user is holding and to which the feedback device isattached. When the directional feedback device generates the forcevector, the user holding an object attached to the feedback device orthe feedback device itself will receive a force as a result of thegenerated force vector.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

The systems, methods and computer readable media embodiments for adirectional feedback device for providing directional force feedback infree space in accordance with this specification are further describedwith reference to the accompanying drawings in which:

FIGS. 1A and 1B illustrate one embodiment of relationships which cangenerate a torque representing a force to a user.

FIG. 2 illustrates an embodiment of a self-contained directional forcefeedback device.

FIG. 3 depicts a toy sword example of a physical object including anembodiment of a free space directional feedback device.

FIG. 4 illustrates an example embodiment of a computing environment fordetermining a physical force vector by an executing application.

FIGS. 5A and 5B illustrate an embodiment of a control system of adirectional force feedback device.

FIG. 6 is a flowchart of a method embodiment defining a force vector fora directional force feedback device.

FIG. 7 is a flowchart of a method embodiment for generating a force withreference to a home position of the device.

FIG. 8A illustrates an embodiment of a free space directional feedbackdevice.

FIG. 8B illustrates another embodiment of a directional feedback device.

FIG. 9A illustrates a pancake motor system housed in one of theattachment structures of FIG. 8B.

FIG. 9B illustrates another pancake motor system housed in another ofthe attachment structures of FIG. 8B.

FIG. 10A illustrates another embodiment of a directional feedback devicein which magnets in an arrangement based on a brushless DC motor rotatethe mass about the axial structure.

FIG. 10B illustrates another embodiment of a directional feedback devicein a different arrangement of the magnets for rotation of the mass aboutthe axial structure.

FIG. 11A illustrates an electromagnet housed in one of the attachmentstructures of FIGS. 10A and 10B.

FIG. 11B illustrates an electromagnet housed in the other attachmentstructure of FIGS. 10A and 10B.

FIG. 12A illustrates yet another embodiment of a directional feedbackdevice.

FIG. 12B illustrates the embodiment of FIG. 12A in other than a homeposition.

FIG. 13A illustrates a directional feedback device which can act as auser input device.

FIG. 13B illustrates an example configuration for sensing changes inconductive gel.

FIG. 14A illustrates another embodiment of a directional feedback devicewhich can also serve as a user input device using at least onedesignated pressure point.

FIGS. 14B and 14C illustrate another example configuration for sensingchanges in the conductive gel with the designated pressure points ofFIG. 14A.

FIG. 15 illustrates a flow chart for an embodiment of a method forprocessing user input from a free space directional feedback device.

FIG. 16 illustrates an example embodiment of a configuration of a targetrecognition, analysis and tracking system with a user playing a swordfighting game software application.

FIG. 17 is a flow chart of an embodiment of a method for providingsensory directional feedback in free space that can operate in a targetrecognition, analysis, and tracking system.

FIG. 18A illustrates a detailed example of an embodiment of a computingenvironment that may be used in a gaming console like that in FIG. 16 inwhich one or more embodiments for providing directional feedback in freespace can operate.

FIG. 18B illustrates another example embodiment of a computingenvironment in which one or more embodiments for providing directionalfeedback in free space can operate.

FIG. 18C illustrates an example embodiment of a networked computingenvironment in which one or more embodiments for providing directionalfeedback in free space can operate.

FIG. 19 illustrates an embodiment of an image capture system for usewith a target recognition, analysis, and tracking system that may beused with one or more embodiments.

FIGS. 20A and 20B show a flowchart of a method embodiment for tracking auser holding a directional feedback device.

FIG. 21A illustrates a sword strike between two virtual swords.

FIG. 21B illustrates a home position of a model of a force generationsystem in a directional feedback device.

FIG. 21C shows a position the axial structure is sent to in response tothe sword strike.

FIGS. 22A and 22B show an alternative embodiment of a feedback deviceformed into a prop.

DETAILED DESCRIPTION

Technology is presented to provide directional force feedback in freespace. Generation of force is provided using a mass which is rotatableabout an axis, with the axis being movable. The mass and movable axisgenerate a force vector having a direction and a magnitude inthree-dimensional space to provide feedback to the user. The forcevector is created in response to an event in application executing in aprocessing device. The force vector represents feedback regarding anevent in the application. The technology generates a force vector bygenerating a torque in the mass. The generated torque magnitude anddirection is felt by user holding the mass. The torque provides thephysical sensation of a force coming from the same direction from whichthe torque is coming.

FIG. 1A illustrates a relationship between a torque, a force and aposition vector for a rotation about a first axis, and FIG. 1Billustrates a relationship between a torque, a force and a positionvector for a rotation about a second axis. Force equals mass timesacceleration (acceleration is the time rate of change). A positionvector “r” represents the distance from a point on the edge of arotating object to the axis of rotation. This relationship is capturedby T=r×F. The magnitude of the torque T is rFsin⊖. ⊖ is the anglebetween the force on the point on the disk's edge and its distance rfrom the axis of rotation. This is 90 degrees in this case so sin ⊖is 1. The magnitude of the torque is determined by the force and theposition vector “r.” All discussions assume a right handed coordinatesystem and application of the right hand rule although a left handedcoordinate system can be used as well with the technology if desired.

FIGS. 1A and 1B illustrate one embodiment of relationships which cangenerate a torque representing a force to a user. FIG. 1A illustrates anaxial structure 104 with a mass 102, in this example, a disk around itscenter of gravity. In this example, the center of gravity is the centerof the xyz reference coordinate system. The axial structure 104 and asshown later in FIG. 1B, the disk 102, rotate about the center ofgravity. In FIG. 1A, the disk is not rotating currently. Force F2 is arotational force causing axial structure 104 to rotate counterclockwisefrom the positive y-axis to the negative x axis, −x axis, to a positionas shown in FIG. 1B. Although the axial structure 104 is rotating, forceF2 is always perpendicular to a point at end 108 at the end of positionvector r2. The torque T2 generated by this rotation is alwaysperpendicular to both the force F2 at the point 108 as it rotates, andthe position vector r2 as represented by T2=r2×F2; torque T2 equals thecross product of r2 and F2. Using the right hand rule, the index fingerpoints in the direction of the position vector r2 (y axis). The middle,ring and little fingers curl in the direction of the rotational forceF2, from the y axis to the −x axis. The thumb points in the direction ofthe torque T2, which in this example, is out of the page along thepositive z axis.

In FIG. 1B, F2 is now zero so the axial structure 104 is aligned with orparallel to the x-axis and stationery. With no force F2, T2 is now zeroas well. In FIG. 1B, the disk 102 rotates about the axial structure 104under a force F1 at the point on the edge of the disk ending theposition vector r1. The force F1 is directed into the page, so using theright hand rule, the torque T1 comes out to the −x-axis, negative xaxis, direction out of end 108.

The force the user is to feel is typically of a finite duration. Forexample, if the torque is to represent a force generated from a contactwith another object in a gaming environment, there would be release fromthe force at the end of the contact. If the contact is a strike, a blowor a hit, it may only last a few seconds at most.

In one embodiment, force F2 can be used to set the direction of thetorque T1 by positioning the axial structure 104 at a certain angle. Inthe examples of FIGS. 1A and 1B, −90 degrees from an initial or homeposition. Once at the angle for the desired direction, force F2 isremoved and force F1 is applied in a certain direction to generate atorque T1 out of the intended end of the axial structure 104. Forexample, if force F1 is directed out of the page, the disk 102 spins inthe opposite direction, and T1 would be directed out of end 106. In oneexample, to minimize the feeling of the torque T2 generated, the axialstructure 104 can be made of lightweight material like a plastic so asnot to contribute too much more mass to the disk. Additionally, theaxial structure 104 can be moved at a speed much, much slower than thedisk speed, thus producing a much, much smaller force F2 and torque T2.For example, the disk speed could be 5000 revolutions per minute (RPM),and the rotation speed of the axial structure could be one tenth that.By keeping the disk spinning longer than the time to direct the axialstructure 104 to the desired angle, the user associates the torque T1with the contact or message being represented rather than the smallertorque T2. In another example, the force F2 can be applied to the axialstructure 104 in a quick, rapid burst followed by duration of a longerforce time period for the spinning disk producing torque T1.

By directing the axial structure 104 to a certain angle and using thetorque generated by the disk 102 rotation, a force vector can berepresented in 360 degrees within a plane defined by two axes, in otherwords along the circumference of a circle.

Having a balanced weighted disk helps keep the perpendicularrelationships intact over time to keep the torque directions consistent.The rate of spin or speed can also be used to control the magnitude ofthe torque and hence the generated force a user senses or feels whenholding the physical object. Increasing the speed increases themagnitude of the torque, and decreasing the speed decreases themagnitude of the torque. Torques are generated via a twist about thesingle allowed axis of rotation coupled with a deflection event.Although the true direction of resultant net torque changes over time,the deflection event occurs with such speed that the experiencedrotational force seems unidirectional to a user.

FIG. 2 illustrates one example of a directional force feedback device.This self-contained directional feedback device 204 comprises a supportstructure 806 enclosed within and attached to a housing, in this examplespherical shell 822. Other shapes of the feedback device can be used.The support structure 806 supports axial structure 804 at both ends asit supports disk mass 802 which is centered about the axial structure.Also in this embodiment, control circuitry 810 (see discussion below) islocated within the disk, and it interprets instructions for forcegeneration received from a computing environment. In response to theinstructions, the control circuitry 810 generates control signals to oneor more force generating systems such as motor systems, for example, ina structure like knob 816, and within the disk 802 itself to rotatestructure 806 and spin the disk 802. The control circuitry 810 alsostops generation of a force in accordance with criteria such as a forcetime period has ended.

The methodology surrounding the withdrawal of force uses deflectionspeed. Upon completion of an impact event such that intended directionalforce is generated, the deflection of the spinning mass ceases, and aslow return to home position is begun. In one embodiment, the massrotates to a home position at about 45 degrees/sec. such that a userexperiences very little discernable torque and the device is ready for asecond impact event in short order.

In this example, spherical shell 822 is translucent to allow displayelements 240 on the disk rim and display elements on the disk surface tobe seen by the user. In other examples, the shell can be transparent. Anexample of a display element is a light emitting diode (LED). Asdiscussed in more detail below, the control circuitry 810 can receivedata via wireless communication from a communicatively coupled computingenvironment such as a gaming console. The data can be for display by theone or more display elements 220, 240. Some examples of data are colorsor images such as compass points or text or video. In some instances, asthe disk rotates, data updated to the display elements can also appearto move. The spinning LED bar would form a visually circular display asthe plate spins at its top speed. The LEDs themselves are preciselytimed to represent all pixels of the display area, as they are spun.

FIG. 3 illustrates a directional feedback device 204 incorporated into aphysical object which may be used to enhance game play. In FIG. 3, thephysical object depicted is a toy sword 200 which incorporates adirectional feedback device 204. The toy sword has a blade 206 and ahandle 202 which extends to the sides. This toy sword may be alightweight, plastic sword.

In this example, a directional feedback device 204 is attached to thesword handle. The attachment can be a simple structural connection suchas a strap with self-adhesive fasteners or snaps. In some examples, thephysical object can be molded to make a space with fasteners or a formfactor into which the directional feedback device fits and snaps inplace. In this example, the feedback device 204 comes with a handleportion 820, which also fits into the sword handle 202. By having astructural form factor for attachment of a physical object, a physicalobject does not need electronic circuitry to interact with the feedbackdevice.

As shown in the example in the drawing, the directional feedback device204 is small enough in diameter that it can be hand held. In oneexample, it is less than four inches in diameter.

FIGS. 4, 5A, and 5B illustrate computing environments communicating in asystem for providing directional force feedback to a user. FIG. 4illustrates an example embodiment of a computing environment includingcomputer hardware and software components for determining a physicalforce vector, and communicating its definition to a control system of adirectional force feedback device such as that illustrated in FIG. 5A.

Computer system 300 comprises one or more processors 304 which, inaddition to at least one central processing unit (CPU), may also includea graphical processing unit (GPU) as the demands of real-time, highmotion audiovisual display may require. In this embodiment, theprocessor(s) are shown having local memory 305 which can embody variouscache designs to assist the processor(s) with the high-speed executiondemands of real-time visual display of complex scenes.

The processor(s) 304 are communicatively coupled with other hardware andsoftware components via a computer communication bus 316. One or morenetwork adapter(s) 306 communicate with one or more networks, includingthe Internet 203 to receive and transmit data for the computer system300. One or more audiovisual controllers 308 (e.g. graphics cards, soundcards) are communicatively coupled to an audiovisual data capture system(e.g. 60 in FIG. 19) as well as processing units in an audiovisualdisplay system (e.g. 56 in FIG. 16). As illustrated in the example ofFIG. 16, the audiovisual display system 56 can be an advanced displaysystem such as a high-definition television (HDTV). In otherembodiments, the display may be a lower resolution display, someexamples of which include a television, a computer monitor, or mobiledevice display.

The computer system has an I/O controller 310 for handling input fromuser input devices 309 such as a keyboard or pointing device (e.g.mouse). One or more removable media interface controllers 307 facilitatethe transfer of data and execution of programs stored on media storagedevices 319 such as DVDs, CD ROMS, removable hard disks, and memorysticks. Memory Controller 312 directs the transfer of data to and fromthe various datastores at the behest of applications 315 executing onthe processor(s) 304.

The computer system 300 or computing environment 300 further includes awireless interface port 333 for sending and receiving data wirelessly.Additionally, the system 300 comprises a sensor interface port 335 forreceiving wirelessly data from remote sensors such as an accelerometer818 (see FIGS. 8A, 8B, 10A, 10B) on the directional feedback device 204.In some instances, an accelerometer may send its data or a controller(508) on the feedback device 204 may send accelerometer data via awireless protocol accepted by the wireless interface port 333. In otherexamples, the protocol can be another wireless protocol such as infraredlight which uses the separate sensor interface port 335.

Memory 314 is representative of the various types of memory present in atypical computer system. These include read-only memory (ROM) for bootsoftware, non-volatile memory for storing the operating system 318 andapplications 315, both system and user space applications. Theapplications 315 include the software and datastores for one or moreforce determination software processing modules 323 used by one or moreapplications 315. Some example of such applications can include gamingapplications, 3D television applications, navigation applications, andeducational applications. The one or more force determination softwareprocessing modules 323 determine a physical force vector which is to begenerated by the feedback device in accordance with criteria for arespective application. In some embodiments, the force determinationsoftware 323 determines the vector definition with respect to theposition data from an accelerometer 818 or other orientation sensingdevices.

The memory 314 is also representative of the volatile storage such asrandom access memory (RAM) in its various technology implementations(DRAM, SRAM, etc.) for use when an application is executing on theprocessor(s) 304.

The various types of memory 314, both non-volatile and volatile, and themedia storage devices 319 are examples of computer-readable storagemedia having encoded thereon computer-executable instructions forperforming a method for providing directional force feedback. Forexample, they can store software and associated data stores, alone or incombination, for a force determination module 323.

The executing applications and modules 323 have access to the operatingsystem 318 and the various information it provides or can access for theapplication such as the port through which data is received or for whichit is destined.

FIG. 5A illustrates an embodiment of a control system 810 of adirectional force feedback device. A wireless communication device 502,in this example, a transceiver 502, receives the wireless signal encodedwith a definition of a force vector, in this example a direction and amagnitude of the desired force vector, from a wireless interface port333 of a communicatively coupled computing environment executing anapplication (e.g. 315). The wireless communication protocol may be RadioFrequency (RF), Bluetooth or one of the IEEE 802 wireless basedstandards (e.g. 802.11 or 802.16 sets of standards) or any othersuitable wireless communication protocol. The transceiver 502demodulates the encoded signal from a carrier wave, or other format ifnecessary. If the signal is not already in digital form, the transceivercircuitry converts the baseband analog signal of the data to a digitalsignal capable of being processed by the controller 508 and otherdigital components.

The digital signal is sent via communication bus 520 to the controller508. Examples of types of controllers 408 include, but are not limitedto, a microcontroller, a microprocessor, or a plurality of such devicesif desired.

Memory 512 is accessible to controller 508. In one example, the memorycan include read only memory (ROM) for storing software executable bythe controller 508 and random access memory (RAM) for use during theexecution of that software. In an embodiment illustrated in FIG. 5B,force processing software application 524 is stored in non-volatilememory, and when executed by the controller 508, determines one or morecontrol messages or signals to send to the force generation modulecontrol hardware 518 to represent the force to be generated by thedirectional feedback device. In one embodiment, a look-up table 526 offorce values may be stored in memory 512 from which the controller 508can select based on the received force definition. In one example, theforce values can include data to cause a motor to rotate its shaft toachieve an angle of rotation or deflection and a speed of rotation. Thedetermined one or more values can be converted to one or more analogsignals by a digital to analog converter 414. In some embodiments, theanalog signal can act as a drive signal to a motor or other forcegenerating mechanism.

As shown in the figures that follow, the feedback device can includeaccelerometers or other orientation sensors 818 which connect through asensor interface port 533 to provide their orientation data for thedevice. In other examples, the sensor 818 transmits its data to thelocal wireless communication device 502. In other examples, the sensorcan also send the data to the wireless interface port 333 of the coupledcomputer environment.

In this embodiment, the memory 512 further comprises sensor data 525 andsensor processing software 527. Various embodiments of the directionalfeedback device 204 include at least one accelerometer, typically a3-axis accelerometer, which gives the orientation of the device 204 withrespect to the ground. In one embodiment, the sensor processing software525 causes the sensor data to be sent to a computing environmentwireless interface port 333 for use by its force determination software323. Definition data for a force vector can be given with respect to anorientation position reference which, for example, can be the positionof the accelerometer on the device 204.

The position data of the accelerometers and/or other orientation sensorscan be stored in position reference data 528 for use by the forceprocessing software 524. The orientation reference point can be anotherarbitrary location on the device 204 other than the location of anaccelerometer. In this case, the relationship between an orientationsensor location and the orientation position reference is stored aswell.

In some embodiments, a home or initial position reference point of theforce generation system 518 is a known position from an accelerometer'slocation on the device. The home position reference point and itspositional relationship with respect to the orientation positionreference is also stored in the position reference data 528. The forceprocessing software 524 uses this information in calculations in orderto represent the requested vector definition with respect to theorientation position reference point's location by a vector definitionwith respect to the home or initial position of the force generationsystem 518. Based on the desired rotations with respect to the homeposition, the force processing software 524 sets the control settings(e.g. values from table 526) for the force generation mechanisms.

Additionally, the control system 810 includes one or more displayelement drivers 529 which receive instructions and some data fromdisplay software 530 executing on the controller 508 for data 530 to bedisplayed on communicatively coupled display elements (220 and 240).Some data may be stored in non-volatile memory of memory 512, and otherdata can be received from an executing application 315 on the coupledcomputing environment 300.

Additionally, the control system 810 can process one or more commandswhich a user can indicate by applying pressure to the feedback device(see FIGS. 13A and 14A). The control system can access a lookup table ofcommands 532 in memory 512 in one embodiment in order to correspondssignals received with specific commands. (see FIG. 15).

The control system is powered via a power bus 510 by a power supply 522.In one embodiment, the power supply is a battery. In one example, thebattery is an inductively charged battery. This is convenient in thatthe directional feedback device can be placed in a wireless charger andcharged. This allows for avoiding wire connections on the directionalfeedback device for charging further supporting self-contained versionsof the device 204. Optionally, the force generation system 518 can drawpower from the inductively charged power supply 522. In anotherembodiment, components of the force generation system 518 may haveinductively charged power supplies located local to the components.

In the examples shown below, the feedback device embodiments have atleast one accelerometer in a known location on the device. A homeposition reference point is also at a known location on a supportingstructure such as a housing like spherical shell 822, and hence at aknown relative position to the at least one accelerometer. In someembodiments, to simplify calculations, an accelerometer can be placed onthe home position reference point.

FIG. 6 is a flowchart of a method embodiment 600 defining a force vectorfor a directional force feedback device. FIG. 7 is a flowchart of amethod embodiment for generating a force with reference to a homeposition of the device. FIGS. 6 and 7 are discussed in the context ofthe computing environment of FIG. 4 and the control system 810 of thefeedback device illustrated in FIGS. 5A and 5B for illustrative purposesonly and not to be limiting thereof.

An application 315 executing on the processor 304 determines a forceproducing event has occurred. For example, user input from the feedbackdevice 204 needs a response; or an instruction such as a navigationalsuggestion to move in a certain direction needs to be communicated to auser holding the device; or a contact has been made with a physicalobject attached to the feedback device held by the user. Responsive to aforce producing event occurring, a force determination software module323 associated with the application determines 602 an orientation of thefeedback device, for example based on data from an accelerometer on thedevice. Based on the force producing event, the force determinationmodule 323 determines 604 the force duration time period. The forcedetermination module 323 determines 606 the force direction of the eventwith respect to an orientation reference position on the directionalfeedback device. In one example, this is the location of one or more3-axis accelerometers on the feedback device. The module 323 can alsodetermine 608 a magnitude of the force to be generated and communicates610 the force definition with respect to the orientation referenceposition to the controller 508 of the directional feedback device 204.In the example of a contact with a physical object being the forceproducing event, an application 315 such as a gaming application canreceive image data of the object, for example, and identify an angle atwhich it is hit. Depending on the degree of resolution, the additionalorientation data from a sensor on the device can help identify motion ofthe physical object, for example, whether the edge of the blade of thesword 200 is horizontal or vertical or somewhere in between. Theorientation data can also reflect motions such as spinning of the objectin a person's hand to a finer resolution. In another example, a user maybe holding the device 204 itself in his hand, and the application 315needs to instruct the user to move to his left. The control system 508needs to determine the relationship between the user's left and wheretorque vectors of the force generation system would be directed.Determining the orientation of the device and having an orientationreference position to start from helps a control system 810 for forcegeneration enclosed within the feedback device 204 determine thedirection in which a force vector should be pointing. Furthermore, theforce generation system 518 of the feedback device 204 has an initialposition or home position as a reference point from which to have astarting point or origin to define angles.

FIG. 7 is a flowchart of a method embodiment for generating a force withreference to a home position of the device. The controller 508 receives702 a force vector definition from a force determination module 323executing in a communicatively coupled computing environment, and it isdefined with respect to an orientation reference position of the device.As mentioned above, the reference position can be the location of anorientation sensor such as an accelerometer on the device. Thecontroller 508 can access from the memory 512 the position 528 of theaccelerometer on the device 204 and the position 528 of the homeposition reference point on the device. The force processing software524 executing on the controller 508 determines 704 any changes to theforce vector definition due to translating its reference from theorientation reference position to a home position reference. In somecases where an accelerometer rests on the home position reference point,there may be little or no changes required in the force vectordefinition. The force processing software 524 determines 706 whether thefeedback device is in home position. If not, the force processingsoftware 824 causes instructions to be sent to the force generatingsystem 518 to return 716 the force generation system 518 to homeposition.

If the device is already in home position, the force processing software524 sends instructions to the force generation system 518 to generate708 the force with respect to the home position. Responsive to forceduration criteria being satisfied 710, the force processing software 524causes 714 the force generation system 518 to withdraw the force andreturn 716 the device to home position. Otherwise, the force generationsystem 518 continues 712 generating the force with respect to the homeposition reference. The following examples illustrating variousembodiments of force generation systems illustrate home positionreferences.

FIG. 8A illustrates an embodiment of a directional feedback device 204.It is illustrated in the context of the toy sword 200 for illustrativepurposes only and not to be limiting thereof. In this embodiment, thedevice 204 has a handle portion 820 and a spherical shell portion 822.The shape of the shell or housing can be any desired shape.

The spherical shell 822 can be part of a support structure supporting aforce generation system. The force generation system comprises thestructures and elements providing power to move the structures to createa force in a designated direction. Attached to the spherical shell 822is an outer support structure 808 attached fixed to the spherical shelland having at least one point of attachment 816 a to an inner supportstructure 806. In this case, the outer support structure 808 has twoattachment structures 816 b and 816 a on opposite sides of the innersupport structure. Within the inner support structure 806, is an axialstructure 804, in this case a shaft or rod, about which a mass, in thisexample a disk 802, rotates. In this example, the magnitude of a forcevector is that of a torque created by controlling the speed of aspinning mass about the axial structure 804. Via the attachmentstructures 816 a and 816 b, motor 814 provides power at least tostructure 816 a to rotate the inner support structure 806 along an axisthat is perpendicular to the axial structure 804. For example, such anaxis can be an imaginary line extending from 816 a to 816 b. Therotation of the inner support structure 806 of the force generationsystem directs the torque generated out of one of the ends of the axialstructure 804 to any angle in the circle of rotation.

In this example, a motor 812 provides the power to spin the disk 802about the axial structure 804 thus producing a torque. Disk 802 isbalanced in weight about the structure 804. By rotating the innersupport structure 806 relative to the outer support and the sphericalshell 822, the torque generated in alignment with one end or the otherof the axial structure can be directed in any of 360 degrees of a circlecentered the disk center and about an axis passing between 816 a and 816b.

In this embodiment, rotation of the inner support structure 806 isreferenced to a home position. Different design choices can select adifferent home position. In the example of FIG. 8A, the device 204 is inhome position when the inner support structure 806 is alignedsubstantially or entirely in the same plane as the outer supportstructure 808. Based on this definition, the device in FIG. 8A is shownin its home position. If the inner support structure 806 were rotatedinto the page, the device 204 would not be in home position, and a forcewould be felt coming from the page out of the axial structure's bottomend when the disk is spinning clockwise. Home position is with respectto the directional feedback device's orientation system enclosed withinthe housing 822, not any physical object which may be attached to thefeedback device. Their orientation systems are independent. Translationbetween them, however, can be done with respect to reference points.

The inner support structure 806 can have a sensor 807 located on its topouter surface that sends data indicating it is aligned with a homeposition reference point 809 on the inner side of the outer supportstructure 808 which is fixed.

At least one sensor, which in the examples of FIGS. 8A, 8B, 10A, 10Bcomprises a three-axis accelerometer 818, is located on the handleportion 820. For a physical object such as a sword, bat, racket, etc.that is hand held, this placement is close to the user's hand andrelatively stationary with respect to the user's hand. An accelerometercan provide orientation data such as pitch and yaw of the physicalobject which can be used to determine motion characteristics for thephysical object. Using the motion characteristics, the one or moreprocessing modules can determine the direction and magnitude of thephysical force vector to be directed on the physical object. Theaccelerometer 818 can wirelessly 819 transmit electrical signals to thecontroller 508 for transmission to a coupled computing environment orthe controller 508 for subsequent transmission.

In this embodiment, electronic control circuitry 810 is housed withinthe disk. For example, it may be implemented as a system on a chip (SoC)including the inductively charged power supply 522. In this example,insulated conductors 823 (e.g. one or more insulated wires) extend fromthe circuitry 810 through the axial structure 804 to the motor 812 forthe disk and via the inner support structure 806 and attachmentstructures 816 a to the motor 814 for rotating the inner structure 806.Motor 814 can direct power via the insulated conductors 823 toattachment structure 816 b in one example. Via the insulated conductors823, the controller 508 can send the control signals indicating thedirection of rotation and the degree of rotation to motor 814 and thedetermined rate of spin of the disk to the motor 812.

FIG. 8B illustrates another embodiment of a directional feedback device204. In this embodiment, the motor 814 in the handle portion 820 isreplaced by one or more small motors in at least one of the attachmentstructures 816 a and 816 b. Furthermore, in this example, the sphericalshell acts as the outer support structure and the attachment structures816 a and 816 b are attached to the spherical shell. To determine homeposition, the inner support structure 806 can still have a sensor 807located on its top outer surface that sends data indicating it isaligned with a reference point 809 except that the reference point 809is on the interior of the spherical shell.

FIG. 9A illustrates a pancake motor system 832 a housed in one of theattachment structures of FIG. 8B, in this case 816 a and FIG. 9Billustrates another pancake motor system 832 b housed in attachmentstructure 816 b. The motors and attachments structures are discussedtogether as they are similarly structured and operate in a similarmanner in this embodiment. A connector 830 a, 830 b for a motor driveshaft 834 a, 834 b in this embodiment extends from the support structure806. The support structure can be plastic, and the connector can bemolded as a protrusion into the attachment structure 816 a, 816 b. Inthis embodiment, a pancake motor system 832 a works in conjunction witha pancake motor system 832 b at the other attachment support 816 b torotate the support structure 806 a desired angle to direct the axialstructure 804 to a desired position. The drive shaft 834 a, 834 b of thepancake motor 832 a, 832 b fits the connector 830 a, 830 b to rotate806. The pancake motor system 832 a, 832 b includes an inductivelycharged battery 833 a, 833 b for providing the motor currents drivingthe shaft.

The pancake motor system 832 a, 832 b can receive control driver signalsfrom the electronic circuitry 810 in the disk via the insulatedconductor 823 as shown in FIG. 8A. However, in this embodiment, thepancake motor systems 832 a, 832 b each include a wireless communicationdevice 835 a, 835 b for receiving control signals from the electroniccircuitry 810 in the disk.

FIG. 10A illustrates another embodiment of a directional feedback device204 in which magnets in an arrangement based on a brushless directcurrent (DC) motor rotate the mass 802 about the axial structure 804.FIG. 10B illustrates another embodiment of a directional feedback devicein a different arrangement of the magnets for rotation of the mass aboutthe axial structure 804.

FIG. 10A illustrates another embodiment of a directional feedback device204. In this embodiment, the motor 812 for rotating disk 802 is replacedwith at least one permanent magnet 1013 in a support beneath the disk802. Electromagnets are located along the attachment supports 816 a and816 b. (See FIGS. 11A and 11B below). FIG. 10B illustrates anotherembodiment of a directional feedback device 204 in which the at leastone permanent magnet 1013 is located on the disk itself.

FIG. 11A illustrates an electromagnet 1014 a housed in one of theattachment structures of FIGS. 10A and 10B. FIG. 11B illustrates anelectromagnet 1014 b housed in the other attachment structure of FIGS.10A and 10B. They are discussed together as they are similarlystructured and operating in this embodiment. Electromagnets 1014 a and1014 b are fixed to their locations. They change their polarity when thecurrent running through them reverses. The center magnet 1013 is apermanent magnet in this example, and it rotates about its center.

The connector 830 a, 830 b for the motor drive shaft 834 a, 834 b is asupport for an electromagnet comprising a metal layer 1042 a, 1042 bencompassed by an insulated conductor 1044 a, 1044 b. For example, aninsulated wire can be wrapped around a sheath of metal. Electroniccontrol circuitry 1040 a, 1040 b is connected to the insulatedconductors 1044 a, 1044 b to monitor timing and reverse the currentpolarity at the appropriate time. The magnet electronic controlcircuitry 1040 a, 1040 b can include an inductively charged battery. Themagnet electronic control circuitry 1040 a, 1040 b can receive controldriver signals from the electronic circuitry 810 in the disk via theinsulated conductor 823 as shown in FIG. 8A. However, in thisembodiment, the magnet electronic control circuitry 1040 a, 1040 b eachinclude a wireless communication device 1045 a, 1045 b for receivingcontrol signals from the electronic circuitry 810 in the disk. Theelectromagnets 1014 a, 1014 b and the pancake motors 832 a, 832 b canalso share a wireless communication device and inductively chargedbattery.

FIG. 12A illustrates yet another embodiment of a directional feedbackdevice 204. In this embodiment, the force generation system comprises anaxial structure 804, about which a mass, a disk 802, spins powered by amotor 812. However, instead of rotating a support structure, a servomotor 1104 is attached to a motor support 1102 which can be attached toor part of the support structure for the device, such as the interior ofthe spherical shell housing 822. The shaft 1112 of the servo motor 1104changes its angular position responsive to changes in a control signalfrom the controller 508. The servo motor 1104 can include a wirelesscommunication device within its control circuitry as well. Servo motorstend to be small, lightweight, and come with control circuitry built in.The shaft 1112 moves one end of a deflector arm 1106 as the shaftchanges angular position. The deflector arm 1106 is connected at itsother end to the axial structure 804. This end tilts the axial structure804 and its centered disk 802 due to an angular position change of theshaft 1112. The tilt of the axial structure 804 is confined by a hinge1108 which also has a support 1110 which can be attached or part of thesupport structure 822 for the directional feedback device 204. FIG. 12Aillustrates the device in its home position in which the deflector arm1106 is perpendicular to the axial structure 804.

FIG. 12B illustrates the embodiment of FIG. 12A in other than a homeposition. The motor shaft 1112 rotates or changes its angle in onedirection causing one end of the deflector arm 1106 to follow itsangular path resulting in the other end of the arm 1106 tilting theaxial structure 804 in the opposite direction to a desired position andto the extent allowed by hinge 1108. In some instances, the tilting ofthe axial structure 804 can be performed in a quick burst.

FIG. 13A illustrates a directional feedback device 204 which can act asa user input device. The spherical shell housing 822 acts as an outershell and encloses an inner shell 1325. Between the shells is conductivegel 1324. The conductivity of the gel is affected by the user's pressureon the device 204. Some examples of conductive gels are silver chloridebased gels and silicone gels. One example of a silicone gel hasconductive particles comprising silver coated mica and oxide free silverflakes. In one embodiment, resistance changes occur in the conductivegel when pressure such as from a hand or finger is applied. Theseresistance changes can serve as signals for commands from a user. Aconductor such as wire 1326 indicates the resistance change via avoltage or current change as voltage (V) equals current (I) multipliedby resistance (R), V=IR.

FIG. 13B illustrates an example configuration for sensing changes in theconductivity of the conductive gel. The wire 1326 connects to a sensorsystem 1304 a which can provide a reference voltage or current to createa circuit in the conductive gel. The sensor detects the current voltageor current, e.g. perhaps periodically, and can wirelessly transmit viawireless communication device 1302 a the value to the wireless device502 for command processing by the controller 508. Commands can berepresented by the amount of pressure applied and the length of time thepressure is applied.

FIG. 14A illustrates another embodiment of a directional feedback devicewhich can also serve as a user input device using at least onedesignated pressure point. In this example, there are five designatedpressure points implemented as five indentations 1402 a-1402 e in thesphere convenient for placing the fingers of a hand. As illustrated, thefeedback device is of a size capable of being hand held. Each designatedpressure point indentation has an enclosed amount 1406 a-1406 e ofconductive gel surrounding it on the side between the outer 822 andinner 1325 shells. A wire 1404 a-1404 e from each enclosed amount of gelfor a respective indentation links its respective indentation to asensor system 1304 a, 1304 b in one of the attachment supports 816 a or816 b. In this way, the number of commands a user can indicate isgreatly increased due to the five pressure points and variouscombinations they allow.

FIGS. 14B and 14C illustrate another example configuration for sensingchanges in the conductive gel with the five designated pressure pointsof FIG. 14A. In this example, a wire 1404 a-1404 b from each of theenclosed amounts 1406 a-b about the indentations 1402 a and 1402 b forthe thumb and index finger indentations are coupled to sensor system1304 a, and a wire 1404 c-1404 e from each of the enclosed amounts 1406c-e about the indentations 1402 c, 1402 d and 1402 e for the middle,ring and little finger indentations are coupled to sensor system 1304 b.All five wires could have been coupled to one sensor system if desired.

Commands will typically be defined for the application context. Forexample, commands can include scroll, open, close, save, exit, click,etc. in a graphical user interface application or environment such asWindows®. In another example, a user may be navigating through ananimated reality of a game, or a three-dimensional display context andmay wish to move his avatar or his view down a certain path. Commandsmay be items such as left, right, forward, backward. An accelerometer818 on the directional feedback device 204 can indicate how fast theuser desires to do so by sensing and forwarding data indicating how fastthe user is turning the device 204 in his or her hand. Additionally, theaccelerometer 818 can provide data representing motion characteristicssuch as the position of the directional feedback device 204 and itsdirection of movement. The controller 508 or a processor 304 of thecoupled computing environment (e.g. 52) can use this information todetermine in which direction the user wants to go.

FIG. 15 illustrates a flow chart for an embodiment of a method 1500 forprocessing user input from a free space directional feedback device. Thecontroller 508 receives 1502 one or more readings from a sensor systemand determines 1504 if the one or more readings correspond to a command,and if so, communicates 1506 the command to a computing environment(e.g. 300). The controller 508 in one example, accesses a look-up tableof stored values in memory 512 and does a comparison. There may be oneor more lookup tables, (e.g. commands 532) relating signals to thefingers and then combinations of finger presses to specific commands.The controller 508 can also monitor the time period a pressure has beenapplied in determining commands. In other embodiments, the readings canbe transmitted directly to a coupled computing environment forprocessing.

Similarly, the directional feedback device 204 can generate a forcevector pointing in a certain direction to indicate to a user a suggestedor commanded direction of movement as indicated by an applicationexecuting in a wirelessly communicatively coupled computing environment.In one embodiment, the controller 508 can process such navigationalcommands as any other force vector.

FIG. 16 illustrates an example embodiment of a configuration of atarget, recognition, analysis and tracking system 50 with a user 58playing a sword fighting game software application executing in acomputing environment 52. In an embodiment the system includes an imagecapture system 60, for example, a camera, that may be used to visuallymonitor one or more users, such as the user 58, such that movementsperformed by the one or more users may be captured, analyzed, andtracked. The movements of the user 58 may be interpreted as controlsthat may be used to affect the application being executed by computerenvironment 52.

In one embodiment, based on the captured image data, the systemrecognizes and tracks the user's natural movements in three dimensionalspace. Using the system, a user's actions can directly control actionsof an associated avatar on a display such as sword fighter avatar 64. Inother words, the avatar 64 can mimic actions of the user 58 inreal-time.

The tracking of user motions to the display of the avatar is preferablyperformed in real time such that the user may interact with an executingapplication in real time. A real-time display refers to the display of avisual representation of a user's motion or pose, wherein the display issimultaneously or almost simultaneously displayed with the performanceof the motion or pose in physical space. For example, an update rate ofa display that echoes a user may be 20 Hz or higher, whereininsignificant processing delays result in minimal delay of the displayor are not visible at all to the user. Thus, real-time includes anyinsignificant delays pertaining to the timeliness of data which has beendelayed by the time required for automatic data processing.

In other example embodiments, the human target such as the user 58 mayhave a physical object such as a toy gun, bat, racket, sword, etc. Insuch embodiments, the user of an electronic game may be holding andusing the object while participating in the game. The motions of theobject are tracked and mapped onscreen so that the avatar is depictedwith a virtual object representing the object that the user ismanipulating. The virtual object tracks the motions of the physicalobject as it is being moved by the user in free space. For example, themotion of a how the user 58 strikes with his sword 200 is tracked andutilized for controlling how his on-screen avatar 64 strikes with hisanimated sword 63.

In one embodiment, the target recognition, analysis and tracking system50 may only track the movements of the physical object 200 that the user58 is holding. Additionally, movements of the physical object or usermay be limited to representation from a certain set of motions or poses.In other words, certain motions or poses trigger action in a game, butnot all natural movements are tracked to the user's avatar.

The target recognition, analysis and tracking system 50 may include acomputing environment 52. The computing environment 52 may be acomputer, a gaming system or console, or the like. According to anexample embodiment, the computing environment 52 may include hardwarecomponents and/or software components such that the computingenvironment 52 may be used to execute applications such as gamingapplications, non-gaming applications, or the like.

According to one embodiment, the target recognition, analysis andtracking system 50 may be connected to an audiovisual device 56 such asa television, a monitor, a high-definition television (HDTV), or thelike that may provide game or application visuals and/or audio to a user58. For example, the computing environment 52 may include a videoadapter such as a graphics card and/or an audio adapter such as a soundcard that may provide audiovisual signals associated with the gameapplication, non-game application, or the like. The audiovisual device56 may receive the audiovisual signals from the computing environment 52and may then output the game or application visuals and/or audioassociated with the audiovisual signals to the user 58. According to oneembodiment, the audiovisual device 56 may be connected to the computingenvironment 52 via, for example, an S-Video cable, a coaxial cable, anHDMI cable, a DVI cable, a VGA cable, or the like.

FIG. 17 is a flow chart of an embodiment of a method for providingdirectional force feedback in free space that can operate in a targetrecognition, analysis, and tracking system. The method can beimplemented as one or more processing modules which can operate bysoftware executing on one or more processors and/or computer hardware oras hardware or firmware. For example, in the computing environments ofthe game console 52 of FIG. 18A, the personal computer environment ofFIG. 18B or the networked computing environment of FIG. 18C, it can beimplemented as software stored and executed as an application program.

In one embodiment, the application interacting with other hardware andsoftware components in its computing environment monitors motion of theuser and her physical object as well as the motion of the avatars andtheir animated objects in the context of the game. In a sword game, forinstance, it monitors contacts between the swords or other physicalobjects within the virtual environment of game. In this way, the one ormore force determination software processing modules 323 determine whena force producing event with respect to a virtual object in the contextof the game has occurred based on motion characteristics for thephysical object under the control of a player. Some examples of motioncharacteristics for the object include position, angle, speed, directionof movement, acceleration, time period of a motion, and a volume ofspace around the user's body in which the physical object moves. Usingthe motion tracking system described above and in the co-pending patentapplications incorporated herein, the above characteristics allow thesystem to control game play by the user with respect to the gameenvironment.

Motion characteristics such as orientation data can also be used tosupplement image data of the object. For example, in the sword gameexample, a target recognition, analysis and tracking system candetermine the position and speed of the object while the accelerometerdata provides supplemental motion characteristics information such asorientation data.

In one embodiment, an application directs 315 that a force be applied tothe directional feedback device in the direction of a force vector on acorresponding virtual object. A force vector is commonly defined interms of a direction and a magnitude. In the embodiment of FIG. 17, theforce determination module 323 determines 1702 a direction to which aforce would be directed on a virtual object in the context of anexecuting application such as a game.

The force can be generated from receiving a contact initiated by anothervirtual object or it can be a reaction force generated when the user hasinitiated a contact with her virtual object. For example, in the swordfight of FIG. 16, the avatar corresponding to the user holds a virtualobject, a sword 63, the movements of which correspond to those of thephysical object, the toy sword 200 held by user 58. In the sword gameapplication, for example, when the other avatar's sword 65 strikes thesword 63 of the user's avatar, force determination software 323executing on the game console 52 identifies at what angle the sword ofthe opponent avatar hits the user's virtual sword.

Determining a force vector can comprise determining a composite orresultant force vector. For example, in the sword game example, eachvirtual sword can strike from different directions, and the avatars canhave the swords locked as they struggle against each other. The opposingswords have opposing forces which effects the direction and magnitudethe user would feel or sense with his physical object. In oneembodiment, the force vector represents the resultant force vector of atleast two virtual force vectors generated on the virtual object in thecontext of the application executing in the computing environment.

At 1704, a determination is made with respect to the magnitude for theforce on the virtual object in the context of an application. The morepowerful a sword blow for example, the stronger the force (or reactionforce) should be felt by the user.

In one or more embodiments, the magnitude of the physical force vectorcan be set proportional to a virtual object's force in the context ofthe executing application, the physical characteristics of the physicalobject or both. In one example, a user is playing with a toy sword, andthe sword is made of lightweight plastic. Some examples of physicalcharacteristics include weight, size and material. The swords in thegame may be represented as heavy steel swords. The magnitude of theforce to be felt by the user holding the sword may be scaled or adjustedto be similar to that of another like plastic sword as a steel swordwould crush a plastic sword. In another context, where a user is using aregular tennis racket similar to one used in actual play, the forcedetermination module 323 can more accurately represent the force thatthe avatar opponent and his or her virtual racket would generate.

In another example, a force magnitude can be scaled to one of a range ofmagnitude values that the directional feedback device is capable ofproducing. For example, in the sword game, user 58 gets a relative senseof how strong a blade strike or blow is depending on the magnitude offorce generated.

Therefore, optionally, the magnitude of the force can be made variableby scaling 1706 the magnitude of the force based on the characteristicsof the physical object, the virtual object or both.

The software causes communicating 1708 of the direction and magnitudefor the force, the force vector, to the directional feedback device 204supported by the physical object 200. The directional feedback devicegenerates a physical force vector based on the definition to create gamefeedback in the device 204 which the user can feel. The software cancommunicate with an operating system that the data representing thedirection and magnitude is to be sent by a wireless adapter (e.g. 333)so that the data can be transmitted wirelessly to the directionalfeedback device. Furthermore, the force determination module 323 canalso communicate a change in the physical force vector to the free spacedirectional feedback device. An example of such a change to be indicatedis that the force no longer applies. For example, the opponent avatar'ssword 65 has lifted from the user's virtual sword 63. Other changes maybe changes in a component vector making up the composite force vector.For example, changes in angles of the blades with respect to each otherwhen the swords remain in contact such as when the avatars are eachapplying a virtual force to their respective swords locked in a contact.

Some embodiments of computing environments for a target recognition,analysis and tracking system which communicates with an embodiment ofthe directional feedback device 204 are described.

FIG. 18A illustrates a detailed example of an embodiment of a computingenvironment 52 that may be used in a gaming console like that in FIG. 16in which one or more embodiments for providing directional feedback infree space can operate. As shown in FIG. 18A, the multimedia console 52has a central processing unit (CPU) 101 having a level 1 cache 103, alevel 2 cache 105, and a flash ROM (Read Only Memory) 107. The level 1cache 103 and a level 2 cache 105 temporarily store data and hencereduce the number of memory access cycles, thereby improving processingspeed and throughput. The CPU 101 may be provided having more than onecore, and thus, additional level 1 and level 2 caches 103 and 105. Theflash ROM 107 may store executable code that is loaded during an initialphase of a boot process when the multimedia console 52 is powered ON.

A graphics processing unit (GPU) 109 and a video encoder/video codec(coder/decoder) 114 form a video processing pipeline for high speed andhigh resolution graphics processing. Data is carried from the graphicsprocessing unit 108 to the video encoder/video codec 114 via a bus. Thevideo processing pipeline outputs data to an A/V (audio/video) port 140for transmission to a television or other display. A memory controller110 is connected to the GPU 108 to facilitate processor access tovarious types of memory 112, such as, but not limited to, a RAM (RandomAccess Memory).

The multimedia console 52 includes an I/O controller 120, a systemmanagement controller 122, an audio processing unit 123, a networkinterface controller 124, a first USB host controller 126, a second USBcontroller 128 and a front panel I/O subassembly 130 that are preferablyimplemented on a module 118. The USB controllers 126 and 128 serve ashosts for peripheral controllers 142(1)-142(2), a wireless adapter 148,and an external memory device 146 (e.g., flash memory, external CD/DVDROM drive, removable media, etc.). The network interface 124 and/orwireless adapter 148 provide access to a network (e.g., the Internet,home network, etc.) and may be any of a wide variety of various wired orwireless adapter components including an Ethernet card, a modem, aBluetooth module, a Radio Frequency module, a cable modem, and the like.Furthermore, the wireless adapter card 148 acts as a wirelesscommunication device such as a transceiver for communicating with thedirectional feedback device 204. The wireless communication protocol maybe Radio Frequency (RF), Bluetooth or one of the IEEE 802 wireless basedstandards (e.g. 802.11 or 802.16 sets of standards) or any othersuitable wireless communication protocol.

System memory 143 is provided to store application data that is loadedduring the boot process. A media drive 144 is provided and may comprisea DVD/CD drive, hard drive, or other removable media drive, etc. Themedia drive 144 may be internal or external to the multimedia console100. Application data may be accessed via the media drive 144 forexecution, playback, etc. by the multimedia console 52. The media drive144 is connected to the I/O controller 120 via a bus, such as a SerialATA bus or other high speed connection (e.g., IEEE 1394).

In one embodiment, a copy of the software and data for one or more forcedetermination modules 323 can be stored on media drive 144 and can beloaded into system memory 143 when executing.

The system management controller 122 provides a variety of servicefunctions related to assuring availability of the multimedia console 52.The audio processing unit 123 and an audio codec 132 form acorresponding audio processing pipeline with high fidelity and stereoprocessing. Audio data is carried between the audio processing unit 123and the audio codec 132 via a communication link. The audio processingpipeline outputs data to the A/V port 140 for reproduction by anexternal audio player or device having audio capabilities.

The front panel I/O subassembly 130 supports the functionality of thepower button 150 and the eject button 152, as well as any LEDs (lightemitting diodes) or other indicators exposed on the outer surface of themultimedia console 52. A system power supply module 136 provides powerto the components of the multimedia console 52. A fan 138 cools thecircuitry within the multimedia console 52.

The CPU 101, GPU 109, memory controller 110, and various othercomponents within the multimedia console 52 are interconnected via oneor more buses, including serial and parallel buses, a memory bus, aperipheral bus, and a processor or local bus using any of a variety ofbus architectures. By way of example, such architectures can include aPeripheral Component Interconnects (PCI) bus, PCI-Express bus, etc.

When the multimedia console 52 is powered ON, application data may beloaded from the system memory 143 into memory 112 and/or caches 102, 104and executed on the CPU 101. The application may present a graphicaluser interface that provides a consistent user experience whennavigating to different media types available on the multimedia console52. In operation, applications and/or other media contained within themedia drive 144 may be launched or played from the media drive 144 toprovide additional functionalities to the multimedia console 52.

The multimedia console 52 may be operated as a standalone system bysimply connecting the system to a television or other display. In thisstandalone mode, the multimedia console 52 allows one or more users tointeract with the system, watch movies, or listen to music. However,with the integration of broadband connectivity made available throughthe network interface 124 or the wireless adapter 148, the multimediaconsole 52 may further be operated as a participant in a larger networkcommunity.

When the multimedia console 52 is powered ON, a set amount of hardwareresources are reserved for system use by the multimedia consoleoperating system. These resources may include a reservation of memory(e.g., 16 MB), CPU and GPU cycles (e.g., 5%), networking bandwidth(e.g., 8 kbs), etc. Because these resources are reserved at system boottime, the reserved resources do not exist from the application's view.

In particular, the memory reservation preferably is large enough tocontain the launch kernel, concurrent system applications and drivers.The CPU reservation is preferably constant such that if the reserved CPUusage is not used by the system applications, an idle thread willconsume any unused cycles.

With regard to the GPU reservation, lightweight messages generated bythe system applications (e.g., popups) are displayed by using a GPUinterrupt to schedule code to render popup into an overlay. The amountof memory required for an overlay depends on the overlay area size andthe overlay preferably scales with screen resolution. Where a full userinterface is used by the concurrent system application, it is preferableto use a resolution independent of application resolution. A scaler maybe used to set this resolution such that the need to change frequencyand cause a TV resynch is eliminated.

After the multimedia console 52 boots and system resources are reserved,concurrent system applications execute to provide systemfunctionalities. The system functionalities are encapsulated in a set ofsystem applications that execute within the reserved system resourcesdescribed above. The operating system kernel identifies threads that aresystem application threads versus gaming application threads. The systemapplications are preferably scheduled to run on the CPU 101 atpredetermined times and intervals in order to provide a consistentsystem resource view to the application. The scheduling is to minimizecache disruption for the gaming application running on the console.

When a concurrent system application requires audio, audio processing isscheduled asynchronously to the gaming application due to timesensitivity. A multimedia console application manager (described below)controls the gaming application audio level (e.g., mute, attenuate) whensystem applications are active.

Input devices (e.g., controllers 142(1) and 142(2)) are shared by gamingapplications and system applications. The input devices are not reservedresources, but are to be switched between system applications and thegaming application such that each will have a focus of the device. Theapplication manager preferably controls the switching of input streamwithout the gaming application's knowledge and a driver maintains stateinformation regarding focus switches. The image capture system 60 maydefine additional input devices for the console 52 (e.g. for its camerasystem).

FIG. 18B illustrates another example embodiment of a computingenvironment 420 in which one or more embodiments for providingdirectional feedback in free space can operate. The computingenvironment 420 comprises a computer 241, which typically includes avariety of computer readable media. Computer readable media can be anyavailable media that can be accessed by computer 241 and includes bothvolatile and nonvolatile media, removable and non-removable media. Thesystem memory 222 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 223and random access memory (RAM) 260. A basic input/output system 224(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 241, such as during start-up, istypically stored in ROM 223. RAM 260 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 259. By way of example, and notlimitation, FIG. 4B illustrates operating system 225, applicationprograms 226, other program modules 227, and program data 228.

The computer 241 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 4B illustrates a hard disk drive 238 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 239that reads from or writes to a removable, nonvolatile magnetic disk 254,and an optical disk drive 240 that reads from or writes to a removable,nonvolatile optical disk 253 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 238 is typically connectedto the system bus 221 through a non-removable memory interface such asinterface 234, and magnetic disk drive 239 and optical disk drive 240are typically connected to the system bus 221 by a removable memoryinterface, such as interface 235.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 4B, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 241. For example, hard disk drive 238 is illustrated as storingoperating system 258, application programs 257, other program modules256, and program data 255. Note that these components can either be thesame as or different from operating system 225, application programs226, other program modules 227, and program data 228. Operating system258, application programs 257, other program modules 256, and programdata 255 are given different numbers here to illustrate that, at aminimum, they are different copies.

In one embodiment, a copy of the software and data for one or more forcedetermination modules 323 can be stored in the application programs 257and program data 255 stored on the hard drive 238 or remotely (e.g.248). A copy 323 can also be loaded as an application program 226 andprogram data 228 in system memory 222 when executing.

A user may enter commands and information into the computer 241 throughinput devices such as a keyboard 251 and pointing device 252, commonlyreferred to as a mouse, trackball or touch pad. Other input devices (notshown) may include a microphone, joystick, game pad, satellite dish,scanner, or the like. These and other input devices are often connectedto the processing unit 259 through a user input interface 236 that iscoupled to the system bus, but may be connected by other interface andbus structures, such as a parallel port, game port or a universal serialbus (USB). The image capture system 60 may define additional inputdevices for the computer 241 (e.g. for its camera system). A monitor 242or other type of display device is also connected to the system bus 221via an interface, such as a video interface 232. In addition to themonitor, computers may also include other peripheral output devices suchas speakers 244 and printer 243, which may be connected through a outputperipheral interface 233.

The computer 241 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer246. The remote computer 246 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 241, although only a memory storage device 247 has beenillustrated in FIG. 4B. The logical connections depicted include a localarea network (LAN) 245 and a wide area network (WAN) 249, but may alsoinclude other networks. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 241 is connectedto the LAN 245 through a network interface or adapter 237. When used ina WAN networking environment, the computer 241 typically includes amodem 250 or other means for establishing communications over the WAN249, such as the Internet. The modem 250, which may be internal orexternal, may be connected to the system bus 221 via the user inputinterface 236, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 241, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 18B illustrates remoteapplication programs 248 as residing on memory device 247. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

The network interface 237 is also coupled to a wireless adapter 262providing a wireless communication device such as a transceiver forcommunicating with the directional feedback device. Again, the wirelesscommunication protocol may be Radio Frequency (RF), Bluetooth or one ofthe IEEE 802 wireless based standards (e.g. 802.11 or 802.16 sets ofstandards) or any other suitable wireless communication protocol.

Each of the illustrated computing system environments, 52, 420 and 470,is only one example of a suitable computing environment and is notintended to suggest any limitation as to the scope of use orfunctionality of the presently disclosed subject matter. Neither shouldthe particular computing environment example be interpreted as havingany dependency or requirement relating to any one or combination ofcomponents illustrated in its respective exemplary operatingenvironment. In some embodiments the various depicted computing elementsmay include circuitry configured to instantiate specific aspects of thepresent disclosure. For example, the term circuitry used in thedisclosure can include specialized hardware components configured toperform function(s) by firmware or switches. In other examplesembodiments the term circuitry can include a general-purpose processingunit, memory, etc., configured by software instructions that embodylogic operable to perform function(s). In example embodiments wherecircuitry includes a combination of hardware and software, animplementer may write source code embodying logic and the source codecan be compiled into machine readable code that can be processed by thegeneral purpose processing unit. Since one skilled in the art canappreciate that the state of the art has evolved to a point where thereis little difference between hardware, software, or a combination ofhardware/software, the selection of hardware versus software toeffectuate specific functions is a design choice left to an implementer.More specifically, one of skill in the art can appreciate that asoftware process can be transformed into an equivalent hardwarestructure, and a hardware structure can itself be transformed into anequivalent software process. Thus, the selection of a hardwareimplementation versus a software implementation is one of design choiceand left to the implementer.

FIG. 18C illustrates an example embodiment of a networked computingenvironment in which one or more embodiments for providing directionalforce feedback can operate. As shown in FIG. 18C, multiple consoles400A-400X or processing devices, such as those illustrated in FIGS. 18Aand 18B may be coupled to a network 402 and can communicate with eachother and a network gaming service 404 having one or more server(s) 406via network 402. The server(s) 406 may include a communication componentcapable of receiving information from and transmitting information toconsoles 400A-X and may provide a collection of services thatapplications running on consoles 400A-X may invoke and utilize.

Consoles 400A-X may invoke user login service 408, which is used toauthenticate and identify a user on consoles 400A-X. During login, loginservice 408 obtains a gamer tag (a unique identifier associated with theuser) and a password from the user as well as a console identifier thatuniquely identifies the console that the user is using and a networkpath to the console. The gamer tag and password are authenticated bycomparing them to a global user profile database 416, which may belocated on the same server as user login service 408 or may bedistributed on a different server or a collection of different servers.Once authenticated, user login service 408 stores the console identifierand the network path in the global user profile database 416 so thatmessages and information may be sent to the console.

In an embodiment, consoles 400A-X may include a gaming service 410, asharing service 412, force determination software 323, and user sharingdata 428. The gaming service may allow users to play online interactivegames, create and share gaming environments for joint game play betweenconsoles, and provide other services such as an online marketplace,centralized achievement tracking across various games and other sharedexperience functions. A sharing service 412 allows users to share gameplay elements with other users. For example, a user on a console 400 xmay create elements for use in games and share them or sell them toother users. In addition, a user may record elements of the game playexperience, such as a movie of a race or various scenes in a game, andshare them with other users. Information provided by users for sharingor sale may be stored in the user sharing data 428.

Besides sending the updated avatar and scene data to all theparticipating client computers, force determination software 323 of thenetwork gaming service 404 can determine force producing events anddetermine the force vectors to be sent to each respective clientcomputer. In a heavy action scene like a battle with many participants,this can help speed processing.

The global user profile database 416 may include information about allthe users on consoles 400A-X such as the users' account information anda console identifier that uniquely identifies a particular console thateach user is using. The global user profile database 416 may alsoinclude user preference information associated with all the users onconsoles 400A-X. The global user profile database 416 may also includeinformation about users such as game records and a friends listassociated with users.

Any number of networked processing devices may be provided in accordancewith a gaming system as provided in FIG. 4. As such, the technologypresented herein may operate on one or more servers 406 in conjunctionwith a gaming service 404 or may be provided in individual processingdevices in a networked environment, such as devices 400A-400 x.

FIG. 19 illustrates an example embodiment of a target recognition,analysis, and tracking system 50 including an image capture system 60that may be used with one or more embodiments. The image capture system60 identifies human and non-human targets in a capture area and tracksthem in three dimensional space.

It includes an image capture component 70 capable of capturing depthdata in addition to color and line data. As shown in FIG. 19, accordingto an example embodiment, an capture component 70 may include an IRlight component 72, a three-dimensional (3-D) camera 74, and a color(e.g. RGB) camera 76 that may be used to capture the depth image of acapture area. Various 3-D techniques can be used to determine depth datawith the infrared (IR) component alone or in conjunction with the datafrom the other cameras. Some examples of such techniques include time offlight analysis, monitoring phase shift of outgoing and incomingsignals, shuttered light pulse imaging, and structured light patternprocessing. The depth data can represent distances of different pointsof an object or human from the capture component.

Color data from the color camera 76 can supplement the information fromthe 3-D camera 74 and IR component 72 to enable a more completerecognition of the human target's movement or position.

According to another embodiment, the capture system 60 may include twoor more physically separated cameras that may view a capture area fromdifferent angles, to obtain visual stereo data that may be resolved togenerate depth information.

The capture system 60 can capture data at interactive rates, increasingthe fidelity of the data and allowing the disclosed techniques toprocess the raw depth data, digitize the objects in the scene, extractthe surface and texture of the object, and perform any of thesetechniques in real-time such that the display (e.g. 56) can provide areal-time depiction of the scene on its display screen (e.g. 54).

In the system embodiment of FIG. 19, the image capture system 60 iscommunicatively coupled 84 to a computing environment 300, in thisexample a multimedia console. The communication coupling can beimplemented in one or more wired or wireless connections such as, forexample, a USB connection, a Firewire connection, an Ethernet cableconnection, or the like and/or a wireless connection such as a wireless802.11b, g, a, or n connection.

The capture system 60 further includes a memory component 82 for storinginstructions that may be executed by the processor 80, as well as imagedata which may be captured in a frame format. The memory component 82may include random access memory (RAM), read only memory (ROM), cache,Flash memory, a hard disk, or any other suitable storage component. Inone embodiment, the memory component 82 may be a separate component incommunication 90 with the image capture component 70 and the processor80 as illustrated. According to another embodiment, the memory component82 may be integrated into the processor 80 and/or the image capturecomponent 70.

The capture system 60 further includes a processor 80 communicativelycoupled 90 to the image capture component 70 to control it and thememory 82 for storing image data. The processor 80 may include astandardized processor, a specialized processor, a microprocessor, orthe like that may execute instructions that may include instructions forstoring profiles, receiving depth image data, storing the data in aspecified format in memory 82, determining whether a suitable target maybe included in the depth image, converting the suitable target into askeletal representation or other type of model of the target, or anyother suitable instruction. The inclusion of processing capabilities inthe image capture system 60 enables a model such as a multi-pointskeletal model, of a user and/or an object to be delivered in real-time.Furthermore, some of this processing may be executed by other processors(e.g. 101, 109, 259, 229, 304, 472) in one or more communicativelycoupled computing environments.

The capture system 60 may further include a microphone 78 which can beused to receive audio signals produced by the user. Thus, in thisembodiment, the image capture system 60 is an audiovisual data capturesystem. The microphone(s) in the capture system may be used to provideadditional and supplemental information about a target to enable thesystem to better discern aspects of the target's position or movement.For example, the microphone(s) may comprise directional microphone(s) oran array of directional microphones that can be used to further discernthe position of a human target or to distinguish between two targets.

Image data is captured iteratively, usually in frames. Differences inthe captured image data are tracked based on the models and changes inthe data. From these differences, a user's natural movements, and themovements of an object like the sword in FIG. 16 are tracked.

The technology is advantageously utilized in a target recognition,analysis, and tracking system such as that disclosed in U.S. patentapplication Ser. No. 12/475,094 entitled “Environment And/Or TargetSegmentation”, filed May 29, 2009 and hereby fully incorporated hereinby reference; U.S. patent application Ser. No. 12/603,437, “PoseTracking Pipeline,” filed on Oct. 21, 2009, and hereby fullyincorporated herein by reference; U.S. patent application Ser. No.12/475,308, “Device for Identifying and Tracking Multiple Humans OverTime,” filed on May 29, 2009, and hereby fully incorporated herein byreference; “Motion Detection Using Depth Images,” filed on Dec. 18,2009, and hereby fully incorporated herein by reference; U.S. patentapplication Ser. No. 12/575,388, “Human Tracking System,” filed on Oct.7, 2009, and hereby fully incorporated herein by reference U.S. patentapplication Ser. No. 12/422,661, “Gesture Recognizer SystemArchitecture,” filed on Apr. 13, 2009 and hereby fully incorporatedherein by reference; and U.S. patent application Ser. No. 12/511,850,entitled “Auto Generating a Visual Representation,” filed 29 Jul. 2009,fully incorporated herein by reference.

Identifying and tracking a target, be it human or non-human, istypically an iterative process. FIGS. 20A and 20B show a flowchart of amethod embodiment 2000 for tracking a user holding a directionalfeedback device in order to determine whether a force producing eventhas occurred. In one embodiment, the processing is performed by anapplication executing on processor 80 in the image capture system orexecuting in computing environment 52 or a combination of both. Forillustrative purposes only and not to be limiting thereof, the methodembodiment is discussed with respect to such application software.

The executing application software receives 2002 notification of thepresence of a directional force feedback device. For example, thecontroller 508 sends a message or orientation data from an accelerometerto alert the application of its presence. The application receives 2005image data including depth data, and determines 2010 whether there is anarea of interest. An area of interest may be a concentration of adjacentpixels having depth values in a very narrow range. The depth data inconjunction with edge detection results and the color data can determinewhether an area of interest fits a pattern for a target type such as ahuman being or a sword or a racket, etc.

The application software applies pattern matching with a model based ona human skeletal form for instance in determining 2015 whether the areaof interest is a human target or not. If it is, the application softwarescans the human target 2020 for body parts and generates a model for thecaptured human target 2030. For example, the software starts at a headarea based on the model, and defines body parts from there such asshoulders based on patterns and updates to the pixel data over time. Ifnot a human target, the application receives 2005 image data on the nextiteration and does the processing continually.

Once a human has been identified, the application determines whether thehuman is holding the feedback device 204 which is present. If not, theapplication continues 2040 tracking and updating the human model and anynew areas of interest as new data is received 2005 with each iteration.

If the human is holding the feedback device, the application determines2050 whether the feedback device is attached to an object. Theapplication can retrieve patterns of the types of objects it uses. Forexample, the application can be a sword fighting application whichrecognizes swords, shields, and items of that nature. The applicationmay have patterns stored for physical objects of certain manufacturersthat are specially made for attachment of a directional feedback device.A pattern of a directional feedback device type can also be applied.Additionally, an object model for the target can also be generated basedon the observed features of the physical object.

If the feedback device in an unattached mode is being held by themodeled human, the application incorporates 2055 the feedback device inthe human model, and tracks 2060 the human model and orientation datafrom the feedback device.

If the feedback device is attached to an object, the application canincorporate 2070 the model of the physical object including the feedbackdevice in the human model. In other words, the human and object can betreated as one model. In other examples, the physical object model orfeedback device model can be tracked separately if preferred, withreference to the human model. The application tracks 2060 the humanmodel and orientation data from the feedback device.

Whether the feedback device alone is held or a physical object isattached to it, responsive to a force producing event, a forcedetermining software module 323 determines a force vector definition andduration for the application based on the model and orientation data.For example, the force determining module 323 can execute a methodembodiment like that of FIG. 6 and communicate the vector definition andduration to the directional force feedback device 204 for generation.

FIG. 21A illustrates a sword strike between two virtual swords. FIG. 21Billustrates a home position of a model of a force generation system 518in a directional feedback device 204. FIG. 21C shows a position theaxial structure 804 is sent to in response to the sword strike.

In FIG. 21A, virtual sword 201 strikes a virtual sword corresponding tophysical sword 200 having attached directional force feedback device204. FIG. 21B illustrates the force generation system 518 of thedirectional feedback device 204 before the contact assuming the sword200 is stationary. The device 204 is in home position in this example.Its current orientation for the flat blade can be represented byconsidering the x axis running from supports 816 a through 816 b. Axialstructure 804 is aligned along a y-axis.

Fsw is the virtual force being directed on the virtual counterpart ofthe physical sword. The force would push the blade 206 of the physicalsword 200 down the y-axis (towards negative y) as well as at an angleapproximately 45 degrees into a plane between the negative y andpositive z axes.

In response to this as shown in FIG. 21C, the control system 810 wouldcause the pancake motors 832 a and 832 b to rotate the axial structure804 so to align with the imaginary force Fsw vector. So the axialstructure 804 would be rotated −45 degrees from the y-axis and extendsfrom the negative y and z axes into a plane defined by the positive yand z axes. The disk is rotated so that the torque is directed into theplane defined by the x and z axes. In other words, the sword 200 torquemagnitude is directed down and to the left of the sword handle 202 as astrike from the upper right would tend to push it.

FIGS. 22A and 22B illustrate another embodiment of a directionalfeedback device 2204. Device 2204 includes an outer housing 2206 formedin the shape of a sword to encase the mechanical and electricalcomponents of the feedback device. A support 2208 provides structuralrigidity for the force generation system, a power source 2010 (housed ina “handle” of the sword) and electrical components 2012 housed in the“blade” section of the sword. In this embodiment, the force generationsystem comprises a spherical rotating mass 802-1 housed in a rotatinginner ring 806-1 which rotates within an outer ring support 822-1. Ringsupport 822-1 is coupled to support 2208. Inner ring 806-1 is coupled toring support 822-1 by bearings 816-1 and 816-2. The ring support 802-1allows the inner ring to rotate about axis N. Spherical mass 801-2 iscoupled to shaft 2220 and rotates about axis N powered by an internallymounted motor 812-1 positioned within the mass 802-1. As viewed in FIG.22A, inner ring 806-1 may rotated about axis M (and moved in thedirection of arrows 2224, 2226) by a motor 2230 and deflectionstructure. The deflection structure includes a cross-beam 2232 isconnected to inner ring 806-1 and by arms 2234, 2236 to crossbeam 2238.Cross beam 2238 is rotated by motor 2230.

The technology may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. Likewise, theparticular naming and division of modules, routines, features,attributes, methodologies and other aspects are not mandatory, and themechanisms that implement the present technology or its features mayhave different names, divisions and/or formats. Furthermore, as will beapparent to one of ordinary skill in the relevant art, the modules,routines, features, attributes, methodologies and other aspects of theembodiments disclosed can be implemented as software, hardware, firmwareor any combination of the three. Of course, wherever a component, anexample of which is a module, is implemented as software, the componentcan be implemented as a standalone program, as part of a larger program,as a plurality of separate programs, as a statically or dynamicallylinked library, as a kernel loadable module, as a device driver, and/orin every and any other way known now or in the future to those ofordinary skill in the art of programming.

The foregoing detailed description has been presented for purposes ofillustration and description. It is not intended to be exhaustive or tolimit the technology disclosed to the precise form disclosed. Manymodifications and variations are possible in light of the aboveteaching. The described embodiments were chosen in order to best explainthe principles of the technology and its practical application tothereby enable others skilled in the art to best utilize the technologyin various embodiments and with various modifications as are suited tothe particular use contemplated. Moreover, it should be noted that thelanguage used in the specification has been principally selected forreadability and instructional purposes. Although the subject matter hasbeen described in language specific to structural features and/ormethodological acts, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thespecific features or acts described above. Rather, the specific featuresand acts described above are disclosed as example forms of implementingthe claims.

What is claimed is:
 1. A method for providing directional force feedbackin free space to a user, comprising: providing a mass rotatable about amovable first axis, the mass and movable axis being responsive to acontrol signal to generate a force vector having a direction and amagnitude in three dimensional space; and rotating the mass about atleast the first axis to generate the force vector responsive to anapplication executing in a processing device, the application undercontrol of the user and displaying events to the user, the force vectorrepresenting feedback regarding an event in the application.
 2. Themethod of claim 1 wherein the rotating comprises rotating the mass aboutthe movable first axis while simultaneously moving the movable firstaxis to generate the force vector.
 3. The method of claim 1 wherein theproviding includes providing the mass centered about an axial structureand which generates a torque magnitude representative of the magnitudefor the force vector by controlling a speed of the mass spinning aboutthe axial structure.
 4. The method of claim 3 wherein the rotatingcomprises generating the direction for the force vector by moving theaxial structure to generate a torque in the direction of the forcevector, the torque having the torque magnitude.
 5. The method of claim 4wherein moving the axial structure about which the mass spins includestilting the axial structure.
 6. The method of claim 3 comprising theadditional step of rotating the axial structure to a desired position.7. The method of claim 3 wherein the mass has a home position from whichan angle of a tilting of the axial structure is referenced.
 8. Themethod of claim 7 wherein the home position comprises the axialstructure being in alignment with a home position reference point.
 9. Ina system including a computer executing an application communicativelycoupled to a directional force feedback device, a method for providingdirectional force feedback to a device held by a user in free space andconnected to the directional force feedback device, comprising:determining when a force producing event has occurred in theapplication; responsive to an occurrence of a force producing eventwhich creates a force having a force vector, determining a duration forthe force vector based on the force producing event; determining adirection for the force vector in three dimensions with respect to anorientation reference position on the device connected to the forcefeedback device; and generating the force vector for the force durationby rotating a mass in the force feedback device, the mass beingrotatable about a first axis and a second axis orthogonal to the firstaxis, the mass rotating to generate the force vector in the determineddirection and for the determined duration.
 10. The method of claim 9wherein determining a force vector with respect to an orientationreference position comprises determining a force direction with respectto the orientation reference position and a force magnitude of the forcevector.
 11. The method of claim 10 wherein the force producing eventrepresents an action in the application which is displayed to the usercontemporaneously with generating the force vector.
 12. The method ofclaim 11 wherein determining a force producing event comprisesdetermining the force producing event with respect to a virtual objectbased on motion characteristics for a physical object to which thedirectional force feedback device is attached.
 13. The method of claim12 further comprising: receiving the motion characteristics includingorientation data from the free space directional feedback device; andusing the motion characteristics for determining the force direction,force magnitude, and duration of the force vector.
 14. The method ofclaim 13 further comprising determining motion characteristics of thevirtual object based on image data of the physical object and using themotion characteristics of the virtual object for determining the forcedirection, force magnitude, and duration of the force vector which is tobe generated by the feedback device.
 15. The method of claim 14 whereinthe force vector has a magnitude, the magnitude is set proportional tothe virtual object's force in a context of the executing application.16. A computer storage device having instructions to direct one or moreprocessors to perform a method for providing directional force feedbackto a feedback device held by a user, comprising: determining when aforce producing event has occurred in execution of an application, theapplication being under control of the user and displaying one or moreevents to the user, a force vector representing feedback regarding theforce producing event in the application; responsive to an occurrence ofa force producing event which creates a force having a force vector,determining a duration of the force based on the force producing event;determining the direction of the force vector with respect to anorientation reference position, the orientation reference position beinga position in the feedback device; and communicating the force vectorand force duration to the device to rotate a mass, the mass rotatableabout a first axis and a second axis perpendicular to the first axis,the rotation of the mass generating the force vector having thedirection and a magnitude in three dimensional space.
 17. The computerstorage device of claim 16 wherein the mass is the mass centered aboutan axial structure, the computer storage medium having instructions todirect at least one or more of the processors to control a forcegeneration module to generate a torque magnitude representative of themagnitude for the force vector by controlling a speed of the massspinning about the axial structure, the method further comprising:determining whether the mass is in a home position; responsive to themass not being in the home position, returning the mass to home positionby aligning the axial structure with a home position reference point;from the home position, moving the axial structure to a position wherean end of the axial structure points in the direction for the forcevector; and rotating the mass about the axial structure to generate atorque representative of the magnitude of the force vector and whichtorque is directed out of the end of the axial structure pointing in adirection orthogonal to the physical force vector.
 18. The computerstorage device of claim 17 wherein the method further comprisesreceiving orientation data from the feedback device; and determining theorientation of a physical object attached to the feedback device basedon the orientation data.
 19. The computer storage device of claim 18wherein the method further comprises determining when a force-producingevent with respect to a virtual object has occurred based on motioncharacteristics for the physical object to which the mass is attached.20. The computer storage device of claim 19 wherein the motioncharacteristics are determined from orientation data from the feedbackdevice coupled to the one or more of processors providing image data ofthe physical object.